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Listserv Distribute Protocol 


Status of this Memo 


This memo provides information for the Internet community. It does 
not specify an Internet standard. Distribution of this memo is 
unlimited. 

Abstract 


This memo specifies a subset of the distribution protocol used by the 
BITNET LISTSERV to deliver mail messages to large amounts of 
recipients. This protocol, known as DISTRIBUTE, optimizes the 
distribution by sending a single copy of the message over heavily 
loaded links, insofar as topological information is available to 
guide such decisions, and reduces the average turnaround time for 
large mailing lists to 5-15 minutes on the average. This memo 
describes a simple interface allowing non-BITNET mailing list 
exploders (or other bulk-delivery scripts) to take advantage of this 
service by letting the BITNET distribution network take care of the 
delivery. 


Introduction 


Running a mailing list of 1,000 subscribers or more with plain 
"sendmail" while keeping turnaround time to a reasonable level is no 
easy task. Due mostly to its limited bandwidth in the mid-80’s, 
BITNET has developed an efficient bulk delivery protocol for its 
mailing lists. Originally introduced in 1986, this protocol was 
refined little by little and now carries 2-6 million mail messages a 
day. In fact, this distribution mechanism implements a general- 
purpose delivery service which can be used by any user of BITNET or 
the Internet. Thus, a simple solution to the "sendmail" turnaround 
problem is to wrap the message and recipient list in a DISTRIBUTE 
envelope and pass it to a BITNET server for delivery. This may not 
be the best possible solution, but it has the advantage of being easy 
to implement. 


In this document we will use the term "production" to refer to the 
normal operation of the mailing list (or bulk delivery application) 
you want to pipe through the DISTRIBUTE service. That is, the 
"production" options are those you should specify once everything is 
tested and you are confident that the setup is working to your 
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satisfaction. In contrast, "test" and "debug" options can be used to 
experiment with the protocol but should not be used for normal 
operation because of the additional bandwidth and CPU time required 
to generate the various informational reports. 


Finally, it should be noted that the DISTRIBUTE protocol was 
developed to address a number of issues, some of them relevant only 
to BITNET, and has evolved since 1986 while keeping a compatible 
syntax. For the sake of brevity, this RFC describes only a small 
subset of the available options and syntax. This is why the syntax 
may appear unnecessarily complicated or even illogical. 


1. Selecting an entry point into the DISTRIBUTE backbone 


The first thing you have to do is to find a suitable site to submit 
your distributions to. For testing, and for testing ONLY, you can 
use: 


LISTSERV@SEARN.SUNET.SE 


For production use, however, you should select a DISTRIBUTE site in 
your topological vicinity: it would make no sense to pass your 
distributions from California to a server in Sweden if most of your 
recipients are in the US. If your organization is connected to BITNET 
and your BITNET system is part of the DISTRIBUTE backbone, this ought 
to be your best bet. Otherwise you will want to contact someone 
knowledgeable about BITNET (or the author of this RFC if you have no 
BITNET users). Make sure to run through the following checklist 
before sending any production traffic to the site in question: 


a. Do you have good connectivity to the host in question? Does the 
host, in general, have decent BITNET connectivity? There are still 
a few sites that insist on using 9.6k leased lines for BITNET in 
spite of having T1 IP access. You will want to avoid them. 


b. Send mail to the server with "show version" in the message body 
(not in the subject field, which is ignored). Is the server running 
version 1.7f or higher? If so, it should not have given you the 
following warning, 


>>> This server is configured to use PUNCH format for mail <<< 


which means that messages with lines longer than 80 characters 
cannot be handled properly. If the software version is less than 
1.7£, the warning will not be present; instead, check the first 
(bottom) "Received:" field. If it does not say "LMail", do not use 
this server as it probably cannot handle messages with long lines. 
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Finally, make sure that the "Master nodes file" is not older 
than 2 months: there are a handful of sites which never update 
their tables due to staffing problems. They cannot be prevented 
from running LISTSERV, but you will certainly want to avoid them. 


c. How big is your workload? If you are planning to use the service 
for more than 10,000 daily recipients, you should get permission 
from the LISTSERV administrator, both as a matter of courtesy and 
to hear about any restrictions or regularly scheduled downtime they 
might have. For instance, some universities might not allow large 
distributions during prime time, or they may have several 
DISTRIBUTE machines and will want to make sure you use the "right" 
one. Send mail to "owner-listserv" at the host in question and 
give an estimate of the amount of daily messages and recipients you 
would like to submit. If your message bounces back with "No such 
local user" or the like, it means the server did not pass the above 
test (b) and you don’t want to use it anyway. 


An index of sites/hosts which have the required configuration, good 

connectivity, keep their tables up to date and have generally agreed 
to provide this service to anyone in their topological area will be 

published separately in the future. 


2. Physical delivery of the DISTRIBUTE request 


The distribution request is delivered via SMTP to the e-mail address 

obtained in step 1 (for instance, LISTSERV@SEARN.SUNET.SE). In fact, 

as long as you can somehow get mail to the server’s host, you can use 
the service; SMTP is just the most convenient way of doing so. 


2.1. Contents of MAIL FROM: field 


You should set the MAIL FROM: field to the address of the person who 
maintains your mailing list or, generally speaking, to the address of 
a human being who can take action in case the message fails to reach 
the DISTRIBUTE server’s host. This is a very rare occurrence. 


2.2. Contents of RCPT TO: field 


The RCPT TO: field points to the server’s address (for instance, 
LISTSERV@SEARN.SUNET.SE). 


2.3. Contents of the RFC822 header 
After the DATA instruction, you must supply a valid RFC822 header 
with a "From:" field pointing to the mailbox that should receive 


notification of delivery problems, bounced mail, and so on. This can 
be the same as the MAIL FROM: field, an address of the type "owner- 
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xxxx@yourhost", etc. DO NOT PUT THE LIST SUBMISSION ADDRESS THERE, 
or you will get mailing loops. 


For testing, the "From:" field should point to your own mailbox, so 
that you get the responses from the server. 


As long as RFC822 syntax is respected, the only field that matters is 
the "From:" field (or "Sender:", "Resent-—From:", etc.). In practice 
this means you can just pipe the distribution request into "mail 
listserv@whatever" and let your mail program build all the headers. 


3. Format of the DISTRIBUTE request 


The body of the message delivered to LISTSERV defines the recipients 
of the distribution and the text (header + body) of the RFC822 
message you want to have delivered. The request starts with a "job 
card", followed by a DISTRIBUTE command, a list of recipients, and 
finally the message header and body. 


3.1. Syntax of the JOB card 


The purpose of the JOB card is to make sure that any spurious text 
inserted by mail gateways or the like is flushed and not erroneously 
interpreted as a command. It can optionally be used to associate a 
"Job name" with the request, in case you want to use tools to assist 
you in processing the notifications you get from the DISTRIBUTE 
servers when running in test mode. The syntax is as follows: 


//jobname JOB ECHO=NO 


"Jobname" can be anything as long as it does not contain blanks, and 
can be omitted. LISTSERV generally ignores case when parsing 
commands, so you can use "job" or "Job" if you prefer. The ECHO=NO 
keyword is required for production use, to suppress the "resource 
usage summary" you would otherwise get upon completion of your 
delivery. You may want to omit it when testing. 


3.2. Syntax of the DISTRIBUTE command 
Below the JOB card, you must supply the following line: 
DISTRIBUTE MAIL 
For production mode, do not specify anything else on that line. When 
testing, you should add ACK=MAIL in order to get an acknowledgement 
confirming the delivery. There are two other useful options: 


DEBUG=YES, which instructs the server to produce a report showing how 
the various recipients will be routed, but without actually 
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delivering the message; and TRACE=YES, which does the same but does 
deliver the message. Before making a "live" test with your actual 
recipients list, you should tack the DEBUG=YES option once to make 
sure you got all the parameters and syntax right, and get a rough 
idea of the efficiency of the distribution (see the section on 
performance). 


3.3. Giving the list of recipients 


The list of recipients follows the DISTRIBUTE line and is specified 
as follows: 


//To DD * 
userl@host1l BSMTP 
user2@host2 BSMTP 
/* 


The two lines starting with a "/" have to be copied as-is. Each of 
the lines in between contains the address of one of the recipients, 
followed by a blank and by the word "BSMTP", which indicates that you 
do not want the header rewritten. There are four restrictions: 


a. The address must be a plain "local-part@hostname" - no name string, 
no angle bracket, no source route, etc. Bear in mind that the 
DISTRIBUTE server is not in the same domain as you: all the 
addresses should be fully qualified. 


b. If the local-part is quoted, it must be quoted from the first word 
on. Technically, RFC822 allows: Joe."Now@Home".Smith@xyz.edu, but 
for performance reasons this form is not supported. Just quote the 
first word to tell LISTSERV to run the address through the full 
parser: you would write "Joe"."Now@Home".Smith@xyz.edu instead. 


c. The local-part of the address may not start with an (unquoted) 
asterisk. You can bypass this restriction by quoting the local 
part and using a %-hack through the server's host: 
wx** JACK***Sjack-ws.xyz.edu"@server-host. 


d. Blanks are not allowed anywhere in the address. 


You can use the pseudo-domain ".BITNET" for BITNET recipients: it is 
always supported within DISTRIBUTE requests. 


3.4. Specifying the message text 


After the last recipient and the closing "/*", add the following 
line, 
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//Data DD *,EOF 


followed by the RFC822 message (header + body) that you want 
delivered. The EOF option indicates that the message header and body 
will extend until the end of the message you are sending to the 
DISTRIBUTE server. If you are worried about extraneous data being 
appended by a gateway, remove the EOF option, add a closing "/*" line 
after the end of the message, followed by a "// EOJ" card to flush 
any remaining text. This, however, will fail if the message itself 
contains a "/*" line; you would have to insert a space before any 
such line. 


4. Examples 
Here is an (intentionally short) example to clarify the syntax: 


Sore cut here ----- 

//Test JOB 

Distribute mail Ack=mail Debug=yes 
//To DD * 

joe@ws-4.xyz.edu BSMTP 
jack@abc.com BSMTP 
jim@tamvml.bitnet BSMTP 
jill@alpha.cc.buffalo.edu BSMTP 
james@library.rice.edu BSMTP 


/* 
//Data DD *,EOF 

Date: Tue, 19 Jan 1993 10:57:29 -0500 
From: Robert H. Smith <RHS@eta.abc.com> 
Subject: Re: Problem with V5.41 

Tos somelist@some.host.edu 


I agree with Jack, V5.41 is not a stable release. I had to fall back 
to V5.40 within 5 minutes of installation... 


Bob Smith 
Note: some of the hostnames are genuine, but the usernames are all 
fictitious. 


You would get the following reply: 


Job "Test" started on 20 Feb 1993 01:09:40 


> Distribute mail ack=mail debug=yes 
Debug trace information: 
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ABC.COM goes to SEARN (213) - single recipient 
ALPHA.CC.BUFFALO.EDU goes to UBVM (027) - single recipient 
LIBRARY .RICE.EDU goes to RICEVM1 (022) - single recipient 
TAMVM1 goes to TAIVM1 (247) - single recipient 
WS-4.XYZ.EDU goes to SEARN (213) - single recipient 
Path information: 

TAIVM1 : UGA RICEVM1 TAIVM1 

UBVM : UGA UBVM 

RICEVM1 : UGA RICEVM1 

(Debug) Mail forwarded to LISTSERV@UGA for 3 recipients. 


(Debug) Mail posted via BSMTP to jack@ABC.COM. 
(Debug) Mail posted via BSMTP to joe@WS-4.XYZ.EDU. 


Job "Test" ended on 20 Feb 1993 01:09:40 


Summary of resource utilization 


CPU time: 0.086 sec Device I/O: 6 
Overhead CPU: 0.045 sec Paging I/O: 5 
CPU model: 9221 DASD model: 3380 


To actually perform the distribution and get an acknowledgement, you 
would change the first two lines as follows: 


mrss Cüt- here =sssss 
//Test JOB Echo=NO 
Distribute mail Ack=mail 


Mail forwarded to LISTSERV@UGA for 3 recipients. 
Mail posted via BSMTP to jack@ABC.COM. 
Mail posted via BSMTP to joe@WS-4.XYZ.EDU. 


Finally, by removing the "Ack=mail" keyword you would perform a 
"Silent" distribution without any acknowledgement, suitable for 
production mode. 
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5. Performance 


The efficiency of the distribution depends mostly on the quality and 
accuracy of the topological information available to the DISTRIBUTE 
server (and, in some extreme cases, on system load). For BITNET 
recipients, the typical turnaround time for reasonably well connected 
systems is 5-15 minutes. Internet recipients fall in two categories: 
those which can be routed to a machine within or close to the 
recipient’s organization (average turnaround time 5-20 minutes), and 
those for which no topological information is available at all. In 
that case the delivery can take much longer, but usually remains 
faster than with a vanilla sendmail setup. At the time being, 
topological information is available for most top-level domains 
outside the US and for many sub-domains of EDU and GOV. 


You can measure the efficiency of the distribution using the 
DEBUG=YES option as explained above. Recipients which get forwarded 
to another server usually get delivered within 5-20 minutes (except 
to poorly connected sites or countries, for which not much can be 
done). Recipients which are handled locally are passed to a local 
SMTP agent whose efficiency depends very much on the amount of 
"burst" queries the local name server can handle in quick succession. 


A number of projects are currently underway to investigate the 
feasibility of improving the quality of the topological information 
available to the DISTRIBUTE servers for the Internet. 
Security Considerations 
Security issues are not discussed in this memo. 
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